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DETAILED ACTION 

1 . The Office withdraws the previous objection to the claims and the previous rejection of 
the claims under 35 USC § 103(a), in light of the amendment. However, the Office sets forth 
new rejections of the claims under 35 USC § 103(a), in light of the amendment. This Action is 
NonFinal. 



Response to Arguments 

2. Applicant's arguments, with respect to the previous rejection of the claim under 35 USC 
103(a) have been fully considered and are persuasive. Therefore, the rejection has been 
withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view 
of the Lowitz reference. 

Also regarding the previous rejections of the claims under 35 USC 103(a), Applicant 
argues on page s 8-9 that the references do not teach "extracting . . . features from . . . time-based 
media using a feature extraction technique . . .", asserts a "plain meaning" of feature extraction, 
and further argues that Klemmer does not specify "any manner of ... feature extraction 
technique". 

The office respectfully disagrees, noting that the references as a whole teach the subject 
matter as claimed. First, at least the Klemmer reference is noted in the top right paragraph of 
page 92 as discussing the creation of MPEG-2 video and making JPEG thumbnails from video 
sources. Additionally, it is noted that there is no evidence on the record to support Applicants 



Application/Control Number: 10/814,844 Page 3 

Art Unit: 2162 

asserted "plain meaning". And, it is also noted that the claim language does not specify any 
particular manner of feature extraction. 

Therefore, the references have been reasonably interpreted as teaching the recited claim 
language. 

It is further noted that any citation to specific, pages, columns, lines, or figures in the 
prior art references and any interpretation of the references should not be considered to be 
limiting in any way. A reference is relevant for all it contains and may be relied upon for all that 
it would have reasonably suggested to one having ordinary skill in the art. In re Heck, 699 F.2d 
1331, 1332-1333, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting In re Lemelson, 397 F.2d 
1006, 1009, 158 USPQ 275, 277 (CCPA 1968)). 

The Office also notes MPEP § 2144.01, that quotes In re Preda, 401 F.2d 825, 159 USPQ 
342, 344 (CCPA 1968) as stating "in considering the disclosure of a reference, it is proper to take 
into account not only specific teachings of the reference but also the inferences which one skilled 
in the art would reasonably be expected to draw therefrom." Further MPEP 2123, states that "a 
reference may be relied upon for all that it would have reasonably suggested to one having 
ordinary skill the art, including nonpreferred embodiments. Merck & Co. v. Biocraft 
Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). 



For at least these reasons, the Office asserts the rejections of the claims as set forth 

below. 
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Information Disclosure Statement 

3. The information disclosure statement filed 8/20/2009 fails to comply with 37 CFR 
1.98(a)(3) because it does not include a concise explanation of the relevance, as it is presently 
understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about the 
content of the information, of each patent listed that is not in the English language. It has been 
placed in the application file, but the information referred to therein has not been considered. 

4. The information disclosure statement filed 4/1 6/2009 has been placed in the application 
file, but the information referred to therein has not been considered. It is unclear what reference 
Applicant is trying to have considered a reference number A72 (two reference numbers appear 
associated on the line for reference number A72). 



Claim Rejections - 35 USC §103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth 
in section 102 of this title, if the differences between the subject matter sought to be patented and the prior 
art are such that the subject matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 
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6. Claims 1-6, 8, 10, 12-22, 25, 27, 31, 33-41 and 44-45 are rejected under 35 U.S.C. 
103(a) as being unpatentable over Scott R. Klemmer et al. ("Books With Voices: Paper 
Transcripts as a Tangible Interface to Oral Histories", CHI 2003 , Fort Lauderdale, FL, Apr. 5-10, 
2003, pp. 89-96, hereafter referred to as "Klemmer") in view of Graham et al. (US Patent No. 
6,369,81 1, hereafter referred to as "Graham") and Lowitz et al. (US Patent No. 5,485,554, 
hereafter referred to as "Lowitz"). 

Regarding independent claim 1 : Klemmer teaches A computer system for generating 
a representation of time-based media, the system comprising: and generating a media 
representation of the time-based media that represents the extracted features; (See Klemmer 
page 92 in the top paragraph of the right column discussing the creating of an MPEG-2 video 
from a video source. See Klemmer page 92 in the top paragraph of the right column discussing 
the making of corresponding JPEG thumbnails. Also, see Klemmer page 92 in the top paragraph 
of the right column discussing the creating of a paper layout from a time stamped transcript.) 
wherein the formatted media representation includes a graphical representation of a timeline 
and a plurality of user-selectable identifiers indicating locations on the timeline corresponding 
to the extracted features. (See Klemmer page 91 in the last paragraph of the right column 
discussing the adding of time code information to the print format. See also page 92 Fig. 3 
showing barcodes linking chronological sections of a book to A/V data and page 92 in the top 
paragraph of the right column discussing the creating of a paper layout from a time stamped 
transcript.) 
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However, Klemmer does not explicitly teach the remaining limitations as claimed. 
Graham, though, discloses and a printer communicatively coupled to the formatting module, 
the printer for: printing the formatted media representation, (See Graham Fig. 4A #408 and 
col. 5 lines 32-50 teaching the printing of a paper reader's assistant document having an 
imprinted thumbnail and teaching additional information as being a "discussion" of user 
interests.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Graham for the benefit of Klemmer, because to do so allowed a 
designer to implement a mechanism to personalize a document for a particular user, as taught by 
Graham in the Abstract. These references were all applicable to the same field of endeavor, i.e., 
annotation of paper documents with electronic information. 

Additionally, Klemmer does not explicitly teach the remaining limitations as claimed. 
Lowitz, though, teaches a feature extraction module for: extracting, using a feature extraction 
technique, features from the time-based media, the feature extraction technique specified by a 
document format specification file; (See Lowitz col. 4 line 57 - col. 5 line 7 discussing the use 
of a specification for trigger-based capture based upon exemplary features such as a time 
interrupt or a scene change.) a formatting module communicatively coupled to the feature 
extraction module, the formatting module for: formatting the media representation according 
to layout parameters specified by the document format specification file; (See Lowitz col. 12 
lines 27-50 in the context of col. 1 1 lines 30-61 teaching the ability of a user to set formatting 
parameters for printable media.) 
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It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Lowitz for the benefit of Klemmer in view of Graham, because to do so 
enabled a designer to implement a cost-effective, flexible device that is independent of the 
printer type for providing printable time-based media data, as taught by Lowitz in the Abstract. 
These references were all applicable to the same field of endeavor, i.e., translation services. 



Regarding claim 2: Klemmer teaches module further comprises content recognition 
software for recognizing features in the time-based media. (See Klemmer page 92 in the top 
paragraph of the right column discussing the creating of an MPEG-2 video from a video source.) 

Regarding claim 3: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses processing logic for controlling a printer driver interface 
associated with the printer. (See Graham col. 6 lines 4-7 discussing the use of Postscript 
printing.) 

Regarding claim 4: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses processing logic for controlling a printer console on the 
printer. (See Graham col. 6 lines 4-7 discussing the use of Postscript printing.) 



Application/Control Number: 10/814,844 Page 8 

Art Unit: 2162 

Regarding claim 5: Klemmer teaches wherein the feature extraction module is further 
adapted to generate the media representation in digital format. (See Klemmer page 92 in the 
top paragraph of the right column discussing the creating of an MPEG-2 video from a video 
source.) 

Regarding claim 6: Klemmer teaches wherein the feature extraction module is further 
adapted to generate the media representation in paper format. (See Klemmer page 92 Fig. 3 
and the top paragraph in the right column teaching a video paper system.) 

Regarding claim 8: Klemmer teaches wherein at least one of the user-selectable 
identifiers comprises a barcode printed on a document displaying the media representation. 
(See Klemmer page 92 Fig. 3 showing the incorporation of barcodes on a video paper system 
document.) 

Regarding claim 10: Klemmer teaches wherein the barcode on the document can be 
scanned to play time-based media associated with the extracted features on a display device. 
(See Klemmer page 89 Abstract and page 92 in the 1 st paragraph under "Hardware" discussing 
barcode scanning and video playback.) 
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Regarding claim 12: Klemmer teaches wherein the graphical representation includes 
audio content displayed as an audio waveform timeline. (See Klemmer page 92 Fig. 3 showing 
barcodes associated with Voices augmented paper transcripts in the context of the bottom 
paragraph in the right column discussing the adding of barcodes to time code metadata 
information.) 

Regarding claim 13: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to user-selected segments of the time-based media. (See Klemmer page 
92 Fig. 3 showing barcode markers associated with Voices augmented paper transcripts in the 
context of the bottom paragraph in the right column discussing the adding of barcodes to time 
code metadata information.) 

Regarding claim 14: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to segments of audio content, the segments being defined by a search 
for particular features within the time-based media. (See Klemmer page 92 Fig. 3 showing 
barcodes associated with Voices augmented paper transcripts in the context of the bottom 
paragraph in the right column discussing the adding of barcode markers to time code metadata 
information.) 
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Regarding claim 15: Klemmer teaches wherein the timeline includes markers along its 
length that correspond to segments of media content, at least one of the markers having text 
information describing a corresponding segment of media content. (See Klemmer page 92 Fig. 
3 showing barcodes and associated textual passages.) 

Regarding claim 16: Klemmer teaches wherein the timeline includes markers along its 
length that each correspond to a segment of the time-based media, at least one of the markers 
having timestamp information describing the segment of the time-based media. (See Klemmer 
page 92 Fig. 3 showing barcodes associated with Voices augmented paper transcripts in the 
context of the bottom paragraph in the right column discussing the adding of barcodes to time 
code metadata information.) 

Regarding claim 17: Klemmer teaches wherein the media representation includes a 
header describing the time-based media. (See Klemmer page 91 bottom paragraph in the right 
column discussing the placement of metadata in a header.) 

Regarding claim 18: Klemmer teaches wherein the formatting module is further 
adapted to format the media representation is generated according to format specifications 
included in the document format specification file. (See Klemmer page 92 top paragraph in the 
right column discussing the creation of a paper layout based upon a transcript.) 
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Regarding claim 19: Klemmer teaches wherein the format specifications included in 
the document format specification comprise a number of user-definable fields specifying a 
format of a graphical representation of the time-based media. (See Klemmer page 92 top 
paragraph in the right column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 20: Klemmer teaches wherein the format specifications included in 
the document format specification comprise a number of user-definable fields specifying a 
layout of the media representation. (See Klemmer page 92 top paragraph in the right column 
discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 21: Klemmer teaches wherein the format specifications included in 
the document format specification comprise a number of user-definable fields specifying the 
media content markers included in the media representation. (See Klemmer page 92 top 
paragraph in the right column discussing the creation of a paper layout based upon a transcript.) 

Regarding claim 22: Klemmer does not explicitly teach the remaining limitations as 
claimed. Lowitz, though, teaches wherein the document format specification comprises a 
number of user-definable fields specifying the feature extraction techniques to apply to the 
time-based media. (See Lowitz col. 12 lines 27-50 in the context of col. 1 1 lines 30-61 teaching 
the ability of a user to set formatting parameters for printable media.) 
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Regarding independent claim 25: Klemmer teaches A method for generating a 
representation of time-based the method comprising: generating a media representation of the 
time-base media that represents the extracted features; (See Klemmer page 92 in the top 
paragraph of the right column discussing the creating of an MPEG-2 video from a video source. 
Also, see Klemmer page 92 in the top paragraph of the right column discussing the making of 
corresponding JPEG thumbnails. See Klemmer page 92 in the top paragraph of the right column 
discussing the creating of a paper layout from a time stamped transcript.) and wherein the 
formatted media representation includes a graphical representation of a timeline and a 
plurality of user-selectable identifiers indicating locations on the timeline corresponding to 
the extracted features. (See Klemmer page 91 in the last paragraph of the right column 
discussing the adding of time code information to the print format. See also page 92 Fig. 3 
showing barcodes linking chronological sections of a book to A/V data and page 92 in the top 
paragraph of the right column discussing the creating of a paper layout from a time stamped 
transcript.) 

However, Klemmer does not explicitly teach the remaining limitations as claimed. 
Graham, though, discloses printing the formatted media representation, (See Graham Fig. 4A 
#408 and col. 5 lines 32-50 teaching the printing of a paper reader's assistant document having 
an imprinted thumbnail and teaching additional information as being a "discussion" of user 
interests.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Graham for the benefit of Klemmer, because to do so allowed a 
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designer to implement a mechanism to personalize a document for a particular user, as taught by 
Graham in the Abstract. These references were all applicable to the same field of endeavor, i.e., 
annotation of paper documents with electronic information. 

Additionally, Klemmer does not explicitly teach the remaining limitations as claimed. 
Lowitz, though, teaches extracting, using a feature extraction technique, features from the 
time-based media, the feature extraction technique specified by a document format 
specification file; (See Lowitz col. 4 line 57 - col. 5 line 7 discussing the use of a specification 
for trigger-based capture based upon exemplary features such as a time interrupt or a scene 
change.) formatting the media representation according to layout parameters specified by the 
document format specification file; (See Lowitz col. 12 lines 27-50 in the context of col. 1 1 
lines 30-61 teaching the ability of a user to set formatting parameters for printable media.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Lowitz for the benefit of Klemmer in view of Graham, because to do so 
enabled a designer to implement a cost-effective, flexible device that is independent of the 
printer type for providing printable time-based media data, as taught by Lowitz in the Abstract. 
These references were all applicable to the same field of endeavor, i.e., translation services. 
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Regarding claim 27: Klemmer does not explicitly teach the remaining limitations as 
claimed. Graham, though, discloses wherein extracting features from the time-based media 
further comprises performing keyword searching on the time-based media. (See Graham col. 7 
lines 50-65 discussing the use of keywords and keyphrases.) 

Regarding claim 31: Klemmer teaches wherein the graphical representation includes 
audio content displayed as an audio waveform timeline. (See Klemmer page 92 Fig. 3 showing 
text data displayed, the particular data chosen to display having been an obvious variant. See 
also page 92 Fig. 3 showing barcodes associated with Voices augmented paper transcripts in the 
context of the bottom paragraph in the right column discussing the adding of barcodes to time 
code metadata information.) 

Regarding claim 33: Klemmer teaches wherein at least one of the user-selectable 
identifiers comprises a barcode printed on a document displaying the formatted media 
representation. (See Klemmer page 92 Fig. 3 showing barcodes printed on a video paper 
document.) 

Regarding claim 34: Klemmer teaches wherein the barcode on the document can be 
scanned to play time-based media associated with the extracted features on a display device. 
(See Klemmer page 92 in the 1 st paragraph in the right column under "Hardware" discussing the 
use of a barcode scanner.) 



Application/Control Number: 10/814,844 Page 15 

Art Unit: 2162 

Regarding claim 35: Klemmer teaches further comprising generating markers along 
the timeline, the markers corresponding to user-selected segments of the time-based media. 
(See Klemmer page 92 Fig. 3 showing barcodes, in the context of page 91 in the bottom 
paragraph in the right column discussing the adding of time code metadata.) 

Regarding claim 36: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers corresponding to features extracted from the time- 
based media. (See Klemmer page 92 Fig. 3 showing barcodes associated with Voices 
augmented paper transcripts in the context of the bottom paragraph in the right column 
discussing the adding of barcode markers to time code metadata information.) 

Regarding claim 37: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers including text information describing the time-based 
media. (See Klemmer page 92 Fig. 3 showing barcodes and associated textual passages.) 

Regarding claim 38: Klemmer teaches further comprising generating markers along 
the timeline, at least one of the markers including timestamp information describing the time- 
based media. (See Klemmer page 92 Fig. 3 showing barcodes associated with Voices 
augmented paper transcripts in the context of the bottom paragraph in the right column 
discussing the adding of barcodes to time code metadata information.) 
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Regarding claim 39: Klemmer teaches wherein printing the formatted media 
representation further comprises printing a header describing the time-based media. (See 
Klemmer page 91 bottom paragraph in the right column discussing the placement of metadata in 
a header.) 

Regarding claim 40: Klemmer teaches wherein printing the formatted media 
representation further comprises generating a representation in digital format. (See Klemmer 
page 92 in the top paragraph of the right column discussing the creating of an MPEG-2 video 
from a video source.) 

Regarding claim 41: Klemmer teaches wherein printing the formatted media 
representation further comprises printing a representation in paper format. (See Klemmer 
page 92 Fig. 3 showing "Books with Voices augmented paper transcripts".) 

Regarding claim 44: Klemmer teaches further comprising applying a barcode 
generation algorithm to render a barcode image including identifier information. (See 
Klemmer page 92 Fig. 3 showing barcodes that link to further information, it being implied that 
such information requires an identifier for the purposes of storing/locating that information.) 

Regarding claim 45: Klemmer teaches further comprising applying a barcode 
algorithm to render a barcode image including timestamp information. (See Klemmer page 92 
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top paragraph in the right column discussing the creation of a paper layout based upon a 
transcript.) 



7. Claims 28-29 are rejected under 35 U.S.C. 103(a) as being unpatentable over Scott R. 
Klemmer et al. ("Books With Voices: Paper Transcripts as a Tangible Interface to Oral 
Histories", CHI 2003 . Fort Lauderdale, FL, Apr. 5-10, 2003, pp. 89-96, hereafter referred to as 
"Klemmer") in view of Graham et al. (US Patent No. 6,369,81 1, filed Sep. 9, 1998 and issued 
Apr. 9, 2002, hereafter referred to as "Graham"), Lowitz et al. (US Patent No. 5,485,554, 
hereafter referred to as "Lowitz") and Ponceleon et al. (US Patent Application Publication No. 
2003/0187642, filed Mar. 29, 2002 and published Oct. 2, 2003, hereafter referred to as 
"Ponceleon"). 

Regarding claim 28: Klemmer does not explicitly teach the remaining limitations as 
claimed. Ponceleon, though, discloses wherein extracting features from the time-based media 
further comprises performing speech recognition on the time-based media. (See Ponceleon 
Abstract and paragraph [0003] discussing the use of automatic speech recognition for 
discovering salient sections in a speech transcription.) 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to apply the teachings of Ponceleon for the benefit of Klemmer in view of Graham and Lowitz, 
because to do so enabled a designer to implement a system to automatically discover salient 
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segments in a speech transcript, as taught by Ponceleon in the Abstract. These references were 
all applicable to the same field of endeavor, i.e., automation of information retrieval. 

Regarding claim 29: Klemmer does not explicitly teach the remaining limitations as 
claimed. Ponceleon, though, discloses wherein extracting features from the time-based media 
further comprises performing event detection on the time-based media. (See Ponceleon 
paragraph [0007] discussing the detection of events occurring in the news.) 
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Conclusion 



8. The prior art made of record and not relied upon is considered pertinent to applicant's 
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